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ABSTRACT 


This  report  describes  a  study  of  estimation  procedures  for 
the  production  lead  tine  required  in  procurement  of  secondary 
items.  The  accuracy  of  production  lead  time  forecasting  is  par¬ 
ticularly  important  in  the  computation  of  variable  safety  levels. 

The  study  covers  analysis  of  the  performance  of  the  present 
method  for  estimating  lead  times  and  an  attempt  to  formulate  an 
improved  methodology  based  on  relationships  existing  between  lead 
times  and  variables  readily  available  to  the  item  manager. 

It  was  found  that  the  present  methodology  results  in  con¬ 
siderable  forecast  error.  Stepwise  regression  was  then  used  to 
develop  modifications  to  the  present  methodology  that  gave  better 
forecasts,  but  significant  forecast  error  remains. 
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SUMMARY 


1.  Problem  and  Background 

Methodology  now  being  used  by  Army  National  Inventory  Control 
Points  for  computation  of  variable  (statistical)  safety  levels  assumes 
that  the  length  of  the  Production  Lead  Time  is  known  and  that  only  the 
variability  of  demand  Within  this  time  need  be  considered  in  the  Variable 
Safety  Level  computation.  This  assumption  was  questioned,  however,  when 
the  ALPHA  system  was  under  development  so  the  Inventory  Research  Office 
undertook  this  study  to  determine  the  validity  of  the  assumption 
and,  If  invalid,  to  develop  improved  forecasting  methodology. 

2.  Purpose  &  Objectives 

a.  Part  1  of  Study 

(1)  Examine  presently  used  techniques  of  forecasting  Production 
Lead  Time  (PLT). 

(2)  Statistically  determine  forecast  error  of  present  system. 

b.  Part  II  of  Study 

(1)  Develop,  if  possiDie,  a  new  mecnoa  or  lorecasting  eui 

i**.. 

that  would  give  better  accuracy. 

3.  Scope  and  Methods 
a.  Part  I  of  Study 

/< 

Computer  records  of  the  U.S.  Army  Aviation  Systems  Commaud  con¬ 
taining  the  actual  time  till  first  significant  delivery  and  the  devia¬ 
tion  (+  or  -)  of  this  time  from  the  forecast  PLT  were  used  as  the 
basic  raw  dnta.  Statistical  analysis  was  done  of  these  transactions 
to  determine  if  these  deviations  were  significant  enough  to  warrant 
further  study. 


v 


Since  Part  I  showed  large  forecast  errors,  several  alternative 
forecasting  models  were  formulated  and  then  tested  on  the  same  data, 
using  stepwise  regression  techniques.  An  important  constraint  on  this 
study  was  that  only  models  incorporating  easily  available  data  were  " 
considered,  since  feasibility  of  implementation  was  of  paramount 
interest. 

4 .  Findings 

a.  Part  I  of  Study 

The  presently  used  method  of  forecasting  production  lead  time 
is  to  average,  by  FSN,  lead  times  experienced  in  the  past.  A  statistical 
bias  of  25.5  days  was  obsei .id;  i.e.,  the  average  estimate  of  lead 

*  i 

time  was  25.5  days  less  the  i  the  average  lead  time  actually  experienced 

N  < 

for  the  contracts  included  in  the  sample  data.  This  by  itself  is  not 

yUww  uvC.*^Pt  .  /  J  . 

Of  significance  is  that  even  after  the  estimates  were 
corrected  for  bias,  average  error  was  87  days.  What  this  means  is  that 
the  difference  between  estimate  and  actual  varied  greatly  from  contract 
to  contract  with  the  estimate  sometimes  being  much  larger  than  actual, 
and  sometimes  much  smaller.  If  the  differences  between  actual  and 
estimate  are  averaged,  without  regard  to  whether  the  cause  was  a  low 
estimate  or  a  high  estimate,  the  average  difference  is  87  days. 

Standard  deviation  of  error  was  122.6  days.  The  distribution 

•n 

of  error,  excluding  outliers,  was  plotted  and  was  close  to  the  Normal 
distribution  in  appearance. 


A  new  method  of  forecasting  PLT  that  gives  significantly  better 
forecast  accuracy  was  found.  This  method 

(1)  supplements  the  completed  procurement  data  available 
for  a  given  FSN  with  available  currently  completed  procurement  data  for 
other  items  in  the  catalog. 

(2)  stratifies  the  data  into  procurement  method  (PM) 

codes . 


(3)  uses  stepwise  regression  technique  to  determine  how 
the  FSN  data  and  the  catalog  data  are  to  be  weighted  to  give  the  best 
forecast  of  the  PLT  for  the  FSN.  This  is  done  for  each  PM  code  grouping 
of  the  data  and  for  all  data  ungrouped. 

(4)  uses  the  results  of  step  (3)  for  one  of  the  PM 

groupings  in  projecting  PLT  when  the  type  of  contract  is  known  in 
iii.Var.C6 <  uses  the  UT.g. .‘eiui.a  wi  oLep  (jy  cUJWe  .o  c  1.1 

when  there  is  no  prior  knowledge  about  the  type  of  contract. 

The  models  developed  require  annual  updating.  A  program  for 
monitoring  forecast  accuracy  and  providing  input  to  the  model  update 
program  could  be  Incorporated  in  ALPHA.  The  model  update  program  already 
exists  and  could  be  run  off-line. 


CHAPTER  I 


DATA  SOURCE  AND  BASIC  ANALYSES 

1.1  Source 

The  data  base  used  in  this  study  was  originally  in  the  form  of 
magnetic  tapes  from  U.S.  Army  Aviation  Systems  Command  Master  Data 
Record  containing  sector  0  (fixed  sector)  and  sector  07  on  all 
items  in  procurement  at  a  particular  point  in  time  (April  19,  1970). 

1.2  Contents 

The  pertinent  fixed  sector  data  includes  FSN,  Procurement  Lead 
Time  Estimate  and  Standard  Unit  Price.  Procurement  Lead  Time  Estimate 
(PROLT)  is  the  total  of  Estimated  Production  Lead  Time  (ELT),  Estimated 
Delivery  Lead  Time  and  Estimated  Administrative  Lead  Time.  Standard 

•ft 

estimates  are  used  for  administrative  lead  time  (90  days)  and 
delivery  lead  time  (30  days)  making  ELT  -  PROLT  -  .120. 

Sector  07  contains  data  on  orders  currently  in  the  procurement 
process;  data  includes  size  of  order,  and  actual  delivery  dates. 

Completed  contracts  are  dropped  from  sector  07.  .  However,  at  any  given 
time,  sector  07  will  contain  multi-delivery  contracts  waiting  for 
completion  of  delivery  schedules;  it  also  can  contain  single  delivery 
contracts  which  have  been  completed  but  which  are  waiting  for  update 
to  sector  09  of  the  AVSC0M  FSN  Master  Data  Record.  In  both  cases, 
actual  production  lead  times  can  be  computed;  for  multiple  delivery 
contracts  production  lead  time  is  defined  as  time  till  first  signi¬ 
ficant  delivery.  The  data  base  reflects  a  high  proportion  of  high 
dollar  (HDV)  items,  for  which  multiple  deliveries  are  used. 

*After  this  work  was  completed,  it  was  brought  to  light  by  Mr.  W.  Higgins, 
USAMC  Headquarters,  that  there  is  an  error  in  this  statement.  The 
error  and  its  impact  on  our  results  are  discussed  in  Section  3.7. 
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Complete  historical  data  found  in  Sector  09  was  not  ured  In  this 
study  because  it  was  Impossible  to  examine  the  error  of  estimate  on 
each  past  procurement.  Since  the  estimated  Production  Lead  Time  is 
continually  updated,  we  were  unable  to  determine  what  the  estimates 
corresponding  to  past  procurements  were.  Even  using  the  data  in  Sector 
07  did  not  completely  avoid  this  problem  as  the  estimate  in  the  Fixed 
Header  is  updated  as  soon  as  a  delivery  date  is  computed,  except  for 
items  bought  sole  source.  This  indicates  that  the  conclusions  made 
in  this  report  may  be  understated,  with  errors  larger  than  reported, 
and  the  value  of  models  to  supplement  past  experience  would  thus  be 
more  impoi cant . 

An  extract  program  was  written  to  create  an  easily  accessible 
data  base.  The  resulting  data  base  includes  145  different  FSC  groups 
with  a  total  sample  size  of  2039  procurement  actions. 

1.3  Analysis 

a.  Basic  data  analysis  consisted  of  computing  various  statistics 
on  actual  production  lead  times,  estimated  production  lead  times,  and 
variables  readily  accessible  to  an  item  manager  (e.g.  standard  unit 
price).  The  statistical  results  of  this  analysis  are  shown  in  the  first 
column  (total  sample  column)  of  Table  1  with  the  variable  designated  as  follows 
PLT  ■  Actual  Production  Lead  Time 
ELT  ■  Estimated  Production  Lead  Time 
SUP  ■  Standard  Unit  Price 


QTY 


Quantity  of  Order 


EXT  “  SUP  x  QTY,  that  is  extended  price  of  order,  a 
measure  of  economic  importance  of  order. 

VARLOG  =  Logarithm  of  whatever  variable  is  specified,  e.g., 
SUPLOG. 

For  the  total  sample  column  in  Table  1  note  the  high  variability 
of  PLT,  and  the  low  correlations  with  PLT  for  all  variables  but  ELT. 

b.  In  an  attempt  to  find  indications  of  underlying  relationships, 
possibly  non-linear  in  nature,  scatter  diagrams  were  plotted,  first 
of  PLT  vs  ELT  (figure  1),  then  of  PLT  vs  the  logarithms  of  Standard 
Unit  Price,  Quantity  of  contract,  and  Extended  Price  of  contract 
(figure  2-4).  As  can  be  seen,  all  plots  exhibit  a  high  degree  of 
scatter,  but  figure  1  shows  the  least  scatter  of  all.  This  is  con¬ 
sistent  with  the  correlation  statistics  shown  in  the  first  column  of 
Table  1. 

^  A  f'l  Annlwe^  a 

Further  analysis  was  done  in  an  effort  to  find  an  explanation  for 
the  high  variability  in  the  data  by  stratifying  the  data  base  by  pro¬ 
curement  method.  The  contracts  in  the  data  base  were  classified  as 
follows: 

Procurement  Method  Code  1:  Already  Competitive 

2:  Is t  Time  Competitive 

f 

3:  Already  Direct  Purchase  Mftr. 

4:  1st  Tine  Direct  Purchase  Mftr. 

*  r. : 

5:  Non-Competitive  (Sole  Source) 

The  same  statistical  analysis  as  described  in  Section  1.3(a)  was 


done  on  each  of  the  five  groups.  These  results  are  also  shown  in 
Table  1.  Again  we  noted  poor  correlations  and  high  variability  of  PLT, 
but  it  is  easy  to  see  that  the  five  groups  are  different  in  the  range 
of  variability.  PM  Code  5  (sole  source)  exhibits  the  most  variable 
behavior. 

Of  particular  importance  are  the  sizeable  differences  in  average 
PLT  between  FM  Codes  1,  2,  4  vs  PM  Code  3  vs  PM  Code  5. 


CHAPTER  II 


PRESENT  FORE CASTING  TECHNIQUES 
2 . 1  Computation  of  ELT 

Estimated  Production  Lead  Time  (ELT)  is  computed  by  AVSCOM  as 

an  average  of  past  experienced  lead  times  on  contracts  for  the  same 
* 

item.  Contracts  completed  more  than  2  years  past  are  excluded  from 
the  average,  and  generally  provisioning  buys  and  expedited  deliveries 
are  excluded.  For  multiple  delivery  contracts,  estimates  are  based 
on  time  till  first  (non-expcdited)  delivery.  For  other  than  sole 
source  contracts,  the  ELT  is  updated  as  the  new  information  becomes 
available.  For  sole  source  contracts,  only  manual  updates  are  made, 
and  for  any  item,  regardless  of  procurement  method,  a  manual  update  can 
override  the  computer  update. 

ITien  no  applicable  history  c£  procurement  exists,  the 
ELT  rosy  be  entered  as  the  minimum  time  allowed  as  follows: 


AVSCOM  FORECAST  LIMITS 

Unit  price  Minimum  Maximum 

PLTMos .  PLTMps . 


$  0.01 

to 

100.00 

3 

12 

100.01 

to 

1,000.00 

4 

14 

1,000.01 

to 

3,000.00 

6 

16 

3,000.01 

to 

5,000.00 

7 

18 

5,000.01 

to 

7,500.00 

8 

19 

7,500.01 

to 

10,000.00 

10 

21 

over  10,000. 

,00 

11 

24 

*The  description  of  Section  2.1  is  based  on  specifications  for  PLT  up¬ 
dating  given  AVSCOM  ADP  by  AVSCOM  Directorate  of  Materiel  Management 
(17  May  69)  as  well  as  direct  personal  contact  with  AVSCOM  personnel. 


An  Inquiry  was  made  to  AVSCOM  for  a  list  of  constraints  on  ELT 
giving  group  type  and  parameter  set.  Only  one  constraint  was  found 
to  exist;  FSC  group  53XX  with  a  CDC  classification  GA  has  a  maximum 
of  240  days  ELT.  Our  data  sample  only  contained  27  of  this  group 
(1.37.)  so  the  possibility  of  constraints  significantly  affecting 
performance  was  not  further  considered. 

2.2  Performance  Statistics 

Three  performance  statistics  are  used  in  this  report. 

Let 

th 

PLT^  *  actual  production  lead  time  for  the  1  contract 
EST^  -  estimated  production  lead  time  for  the  i  contract 


n  »  number  of  contracts  for  which  statistic  is  computed 

i  Then 

1  n 

(la)  Mean  Error  (ME)  (PLT.  -  EST.) 

n  1-1  11 


*  For  analysis  of  Chapter  2,  EST.  =  ELT.  (l.e. 
estimate).  L 

6 


the  AVSCOM  historical 


Mean  error  was  termed  "statistical  bias"  in  the  report 
summary,  and  mean  absolute  deviation  was  termed  "average  error"  - 
in  order  to  keep  the  summary  non-technieal.  Note  that  mean  absolute 
deviation  and  standard  deviation,  in  effect,  compare  PLT\  to  (EST^  +  ME) 
so  that  errors  are  computed  after  adjustment  for  bias,  i.e.,  after  ME 
Is  added  to  EST^,. 

2.3  Performance  Achieved 

The  performance  statistics  were  computed  for  all  contracts  in 
the  data  base,  treating  all  data  as  one  sample,  and  then  stratifying 
the  data  into  five  samples  by  PM  code.  Results  are  given  in  Table  2. 

Additional  statistics  were  then  tabulated.  A  frequency  distribu¬ 
tion  of  error  was  set  up  by  subtracting  ELT  from  PLT  and  putting  the 
resultant  deviation  into  30  day  classes  (-early,  +  late)*.  Results 

are  shown  in  Figure  5.  Analysis  was  also  done  on  the  relative  error 

PLT -ELT  ,r" 

with  respect  to  the  actual  lead  time  ( — '”) »  A  frequency  dis¬ 
tribution  was  formed  and  me  results  are  shown  in  Figure  6. 


♦Distribution  was  truncated  to  eliminate  outliers;  all  error  values 
outside  of  the  range  +  300  days  were  excluded. 


CHAPTER  III 


N'KW  FORECASTIN’C  MODELS 

3 . 1  Approach 
Technique 

Regression  analysis  was  used  in  the  search  for  an  improved 
methodology  for  forecasting  production  lead  times.  A  regression 
analysis  provides  a  relationship  between  a  dependent  variable  (e.g.  PLT) 
and  one  or  more  independent  variables  (e.g.  standard  unit  price, 
quantity  on  order,  etc.).  Since  there  were  several  independent 
variables,  stepwise  regression  was  used  in  which  the  regression  pro- 

•ff 

gram  (BMD  02R)  proceeds  in  a  stepwise  fashion  forming  successive 
regression  analyses.  A  single  variable  is  added  to  the  regression 
equation  with  each  step  —  the  variable  which  gives  the  greatest 
improvement  in  estimate  —  and  this  continues  as  song  as  statistically 
significant  improvement  can  be  made.  Improvement  is  measured  in  terms 
of  minimizing  the  "biar^cC  error  cf  estimate"  srd  efe.ti-rjc* 1 
significance  is  indicated  by  an  "F"  test.  For  the  regression  equa¬ 
tions,  Standard  Error  of  Estimate  (S.E.)  is  essentially  equal  to  the 
Standard  Deviation  of  the  error  distribution  (equation  lc),  Section  2.2) 
The  stepwise  regression  program  also  computes  Multiple  Coefficient  of 
Correlation  ("Multiple  RM) . 

Experimental  Design 

The  data  base  was  first  separated  into  five  data  segments  by 


*  BMD  02R  Stepwise  Regression  Program,  version  of  May  16,  1968. 

Described  in  detail  in  Biomedical  Computer  Programs;  University  of 
California  Press,  Berkeley  and  Los  Angeles,  1968. 

**The  denominator  has  number  degrees  of  freedom  instead  of  (n-l). 

In  current  context  they  are  very  close. 

8 


> 


** 


PM  code,  and  the  regression  technique  was  then  used  to  explore  each 
segment  individually.  The  data  analysis  of  Section  1.4  (Table  1) 
and  performance  analysis  of  Chapter  2,  summarized  in  Table  2,  indicated 
that  such  a  stratification  was  meaningful. 

Next,  for  completeness  and  possible  ease  of  implementation, 
groupings  were  considered.  That  is,  combinations  of  data  segments 
and  the  data  base  as  a  whole  were  also  subjected  to  the  regression 
analysis.  From  the  results  to  the  above,  prospective  models  were 
chosen  and  their  performance  compared  to  that  of  ELT. 

3.2  Regression 

For  the  reader's  convenience,  abbreviations  for  variables  used 
in  the  regression  equations  are  repeated  here: 

PLT  ■  Actual  Production  Lead  Time 

ELT  •*  AVSCOM  Estimate  of  Production  Lead  Time 

SUP  ■  Standard  Unit  Price 

QTY  «  Quantity  of  Contract 

-  _  clip  V  ATV 

VARLOG  =  Logarithm  of  whatever  variable  is  specified, 

e.g.  SUPLOG.  -i- 

w  Regression  results  were  quite  promising,  both  with  stratification 

/' 

by  PM  code  (Table  3)  and  by  groupings  (Table  4).  In  PM  codes  1,  2,  and 
4,  the  Multiple  Correlation  Coefficient  (R)  was  brought  to  over  .7 
and  the  significance  level  (F-test)  was  over  997.  in  each  case.  PM 
codes  3  and  5  have  somewhat  lower  Multiple  R. 


Regression  results  for  groupings  of  PM  codes  were  not  as  good 


but  still  showed  some  promise.  When  all  PM  codes  are  grouped  together, 
a  Multiple  R  of  over  .5  was  still  achieved  along  with  standard  error 
of  113  days.  To  refer  to  the  four  groupings  in  Table  4,  use  the 
following: 

PM  Codes  Crouped 

1  &  2  -  Competitive 

3  &  4  -  Direct  Purchase  Mftrs. 

1  &  3  -  Already  in  the  Classification 

2  &  4  -  1st  Time  in  the  Classification 

It  is  somewhat  surprising  to  note  that  better  results  come  from 
the  "competitives"  than  from  the  "Direct  purchase  Mftrs."  But  even 
more  notable  is  the  fact  that  models  for  items  already  in  a  classifica¬ 
tion  showed  substantially  less  promise  than  did  the  models  for  the 

j) 

"1st  time"  item  category.  '  // 

j  - n 


The  models  chosen  for  further  accuracy  testing  are  designated 
by  an  asterisk  (*)  in  Tables  3  and  4.  Up  to  three  models  per  PM  code 
or  grouping  were  selected.  These  include: 

,  "Simple"  -  Forecast  model  uses  a  constant  plus  ELT 
variable  only. 

"Price"  -  Forecast  model  uses  a  constant,  ELT,  SUP  and 

V '  J 

SUPLOC  variables  only  (may  not  include 


"Complex"  - 


both  SUP  CjJd  SUPLOC). 

Most  promising  complex  equation  for  the  PM  code 

or  grouping.  Might  include  any  of  the 

variables  studied. 
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"Price"  equations  were  examined  ir.  hope  that  they  could  achieve 
approximately  the  accuracy  of  the  complex  models,  but  with  a  uniform 
set  of  variables  for  all  PM  codes.  It  was  noted  that  unit  price  and 
log  of  unit  price  tended  to  be  statistically  significant  as  demonstrated 
by  their  inclusion,  in  most  cases  when  using  the  automated  stepwise 
regression.  Their  contribution,  however,  in  reducing  error  was 
relatively  small  as  compared  with  ELT. 

3.4  Model  Testing 

The  mean  error  (ME),  mean  absolute  deviation  of  error  (M.A.D.) 
and  standard  deviation  of  error  (S.D.)  were  used  to  compare  the 
AVSCOM  ELT  estimate  and  the  various  regression  models  designated  for 
testing.  Results  are  given  in  Table  5.  (%)  denotes  %  improvement 

of  regression  model  vs  use  of  estimate  equal  to  ELT  only. 

Not  shown  in  Table  5  ie  that  mean  error  was  reduced  to  essentially 
aero,  tfhile  this  is  to  be  expected  when  using  a  regression  equation 
w.;  cl.*,  uuw  -Jut..  Iiu.ii  wii i vJ i  it  was  derived,  we  believe  it  lixely  that 
use  of  the  models  would  reduce  mean  error  in  a  real  world  context. 

In  nearly  all  cases,  significant  improvement  was  found  in  both 
Standard  Deviation  and  Mean  Absolute  Deviation  of  Error.  These 
improvements  over  ELT  arc  legitimate  improvements  in  that  both  per¬ 
formance  measures  were  defined  (cf.  Section  2.2)  to  be  unaffected  by 
the  size  of  mean  error. 

In  Table  5  even  the  simple  model  showed  worthwhile  improvement 
over  use  of  ELT  as  the  estimate;  e.g.,  when  the  simple  model  is 
applied  to  the  data  baee  as  a  whole,  improvement  is  in  the  5-67.  range, 


apart  from  potential  reduction  in  bias  (mean  error).  The  more  complex 
models,  however,  when  applied  by  PM  code,  performed  better  than  the 
simple  models,  with  percent  improvements  in  the  range  of  7.4% 

(PM  Code  5)  to  15.4%  (PM  Code  4).  When  the  PM  code  is  not  known 
in  advance,  the  complex  model  offers  small  improvement  over  the 
simple  model  (cf  results  for  "all"  grouping). 

When  the  8tepwl.se  regression  program  did  not  include  the  price 
variables  for  a  particular  PM  code  or  grouping,  the  program  was  run 
with  a  special  feature  by  which  these  variables  could  be  forced  into 
the  equations.  The  results  of  such  runs  are  not  shown  because  in  no 
case  did  the  "price"  equation  thus  developed  show  real  improvement 
over  the  corresponding  simple  equation. 

3.5  Analysis  of  Simple  Model 

The  simple  model  is  of  the  form: 

(2)  Estimate  «  a  i-  o  x  ELT 

where  a,  b  are  constants  determined  by  regression.  Since  no 
variables  are  used  other  than  ELT ,  the  question  arises  as  to  why  the 
estimates  obtained  should  be  better  than  those  obtained  by  setting 
estimate  “  ELT. 

It  is  shown  at  the  end  of  this  section  that  the  regression  model 
computes  a  and  b  so  use  of  Equation  2  is  equivalent  to  use  of: 

(3)  Estimate  =  (1-w)  ^  +  a  (ELT+B) 


where 

p,  «  mean  production  lead 
B  °  (sample)  bias  in  use 


(• 

time  for  all  contracts  in  sample 
of  ELT  as  estimate  of  PLT 


By  equivalence  is  meant  that  equations  (2)  ar.d  (3)  always  have  the 
same  value.  An  interpretation  of  the  simple  model  is  suggested  by 
equation  (3). 

Two  different  unbiased  estimates  of  production  lead  time 
would  be:  . 

Estimate  •  ELT+B 
♦Estimate  ™  p, 

Furthermore,  the  two  estimates  are  statistically  independent.  ELT  is 
based  on  past  experience  for  one  FSN,  while  ^  is  essentially  inde¬ 
pendent  of  that  history  since  it  is  a  sample  mean  based  on  the 
current  experience  for  a  large  number  of  FSNs.  Now,  it  is  a  well 
known  statistical  property  that  a  weighted  average  of  unbiased 
independent  estimators  can  always  be  found  which  is  superior 
(has  amaller  variance)  to  either  estimator  used  alone.  The  simple 
mod^l  5.8  a  weighted  ®ver“£e  iT.b^ independent  eetirr.ntcrc , 
although  this  is  not  obvious  in  the  form  of  equation  (2).  In  other 
words,  use  of  the  simple  model  is  equivalent  to  modifying  the 
estimate  based  on  ELT  by  the  contract  experience  for  other  FSNs. 

Proof  of  Equivalence  of  equations  (2)  and  (3) 

(i)  b  «*  or  ;  by  definition 

-  fi 

f.<  j '  .  ^ | 

(ii;^;  n  *  a  +  b  x  ELT;  where  ELT  is,  for  all  contracts  in  sample, 
mean  ELT.  This  is  a  well  known  property  of  linear 
regression. 

(ill)  n  «  ELT  +  B  ;  by  definition  of  B. 

*H  is  unbiased  in  that  if  used  as  an  estimator  for  an  item  selected 
randomly,  expected  mean  error  would  be  zero. 
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(iv)  a  =  (x-bx  ELT  ;  from  (ii),  by  algebra. 

(v)  a  =  p  -  a  x  (ji-B)  =  p  x  (1-cr)  +  C'  x  B;  substituting 

(i),  (iii ) ,  (iv)  into  (ii) 

(vi)  a  +  b  x  ELT  “  p  x  (1-a)  tcrxB  +  a  x  ELT  *  (1-cr)  p  +  a 

(ELT+B);  substituting  (i),  (v)  into  (2) 

Q.E.D. 

3 .6  Cone lusions 

a.  Significant  error  exists  about  the  forecast  mean;  this 
forecast  error  should  be  taken  into  account  in  NICP  Safety  Level 
computations. 

.  b.  Forecast  error  can  be  reduced  by  combining  the  individual 
FSN's  past  experience  with  that  of  the  other  FSNs  in  the  catalog. 

Even  better  improvement  can  be  had  if  the  Procurement  Method  is 
Known  in  advance,  stenwi.se  rnorecpioo  technics?  ere  usef"!  for 
determining  both  the  independent  variables  to  use  and  the  form  of  the 
regression  equations. 

c.  Since  conditions  affecting  the  length  of  Production  Lead 
Timea  are  subject  to  change,  analyses  like  those  done  in  this  project 
should  be  done  periodically,  say  annually.  The  ALPHA  system  should 
incorporate  a  program  for  extracting  data  needed  for  analysis.  The 
actual  BMD  02R  stepwise  regression  analysis  can  be  done  off-line. 

d.  An  "order  of  magnitude"  analysis  was  oade  to  estimate  the 

significance  of  a  57.  reduction  in  procurement  lead  time  forecast  error. 

It  was  concluded  that  total  NICP  safety  levels  could  be  reduced  in  the 
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rai'/fie  of  0.57.  to  4.07,  with  no  loss  of  supply  performance,  given  a 
57,  reduction  in  procurement  lead  time  forecast  error.  Since  total 

NICP  apportionment  year  safety  level  requirements  were  176.8  million 

•k 

dollars,  based  on  FY  71  "Strat".  even  a  0.5%  reduction  is  significant. 
It  should  be  recognized,  of  course,  that  this  figure  is  used  Just  to 
determine  whether  it  is  worth  implementing  the  improved  forecasting 
method.  It  has  no  operational  significance  3ince  other  factors  such 
as  the  manner  in  which  statistical  safety  level  techniques  are 
applied,  the  supply  performance  targets  to  be  met,  etc.,  have  to  be 
considered  in  deciding  on  quantitative  changes  in  safety  level  invest¬ 
ment.  In  general,  it  suffices  to  say  that  improvement  in  the  accuracy 
of  the  PLX  forecast  will  enhance  the  accuracy  of  prediction  of  supply 
performance  one  expects  to  achieve  from  a  given  investment  and  that 
the  degree  of  improvement  offered  by  the  new  methodology  is  sufficiently 
large  to  make  its  adoption  worthwhile. 

3.7  Validity  of  Results 

The  process  for  calculating  ELT  discussed  in  section  1.2  was 
based  on  the  belief  that  AVSCOM  used  a  90  day  standard  for  administra¬ 
tive  lead  time.  In  fact,  as  pointed  out  by  W.  Higgins  of  AMC  Head¬ 
quarters,  the  average  estimate  used  was  about  75  days;  subsequent: 
conversation  with  AVSCOM  personnel  indicated  that  the  90  day  standard 

was  used  only  for  the  sole  source  items  (PM  code  5). 

,^-*1 

What  this  means  is  that  for  PM  codes  1-4,  the  values  which 
★Figures  furnished  by  Mr.  W.  Higgins,  AMC. 
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should  have  been  used  for  ELT  in  this  study  were  an  average  of 
something  over  15  days  greater  than  the  values  actually  used.  It 
is  very  unlikely  that  this  error  significantly  affected  the  results 
since  in  evaluating  performance  all  ELT's  used  were  first  upward 
adjusted  anyway  to  remove  statistical  bias  (section  2.2),  and  since 
average  error  after  adjustment  was  87  days. 

In  general,  since  the  sample  used  was  not  random  —  having  a 
predominance  of  multi-delivery  items  —  and  since  the  sample  repre¬ 
sented  only  one  NICP,  at  one  point  in  time,  exact  numerical  results 
should  not  be  extrapolated  to  other  context.  Thus,  these  particular 
regression  coefficients  should  not  be  used  at  other  NICPs  or,  for 
that  matter,  at  the  same  NICP  without  periodic  updating.  The  general 
applicability  of  the  methodology,  however,  is  not  affected  by  the 
special  characteristics  of  the  particular  data  base  used  in  this 
analysis  and  the  conclusions  of  section  3.6  are  eoiioideied  valid. 
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FREQUENCY  DISTRIBUTION  OF  FORECAST  ERROR  (ACTUAL  MINUS  ESTIMATE  LEAD  TIME) 


TABLE  1 


lo.  PSN’s 
;  PLT  (days) 

|  Mean 

j  Std  Dev 

;  ELT  (days) 

'  Mean 

Correlation  Coe.fs 
PLT  vs  ELT 
PLT  va  SUP 
PLT  vs  SUPLOG 

PLT  vs  QTY 

PLT  va  QTYLOG 

i  PLT  vc  EXT 

PLT  vs  EXT LOG 


SUMMARY  OF  ANALYSIS  OF  VARIABLES 


Total 

Sample 


By  Procurement  Method  (PM) 


PM  1 
Already 
Compet 


PM  2 

1st  Time 
Compet 


PM  3 
Already 
Direct 
Purch 


PM  4 
1st  Time 
Direct 
Purch 


PM  5 
Sole 
Source 


2039 

226 

134 

201 

.510 

.021 

.275 

.025 

.028 

.175 

.053 


159 

161 

111 

143 

.688 

.625 

.412 

.080 

.083 

.234 

.154 


105 

183 

91 

164 

.661 

.230 

.466 

.151 

.234 

.117 

.328 


1176 

238 

113 

216 

.427 

.019 

.191 

.007 

.036 

.091 

.045 


317 

161 

106 

143 

.693 
•  111 
.124 
.048 
.319 
.242 
.315 


282 

301 

201 

247 

.318 

.243 

.388 

.032 

.010 

.208 

.075 


TABLE  2 


SUMMARY  OF  ERROR  STATISTICS  BY  PROCUREMENT  METHOD  CODE 


PM  CODE 

MEAN  1 2 

ERROR  OF 
ESTIMATE 

mean(2> 

ABSOLUTE 

DEVIATION 

stand/' 

DEVIATION 
OF  ERROR 

All 

26 

86.9 

122 

1 

18 

59.0 

84 

2 

18 

50.3 

71 

3 

22 

87.0 

114 

A 

18 

62.2 

90 

5 

55 

148. 2 

196 

(1)  Expressed  in  days  late.  & 

(2)  Expressed  in  days. 
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TABLE  3 

1  RESULTS  TO  BMP  -  STEPWISE  REGRESSION 

INDIVIDUAL  PM-CODES 


PM 

STEP 

MULT. 

r< 

EQUATION* 

STD. 

ERROR 

F 

RATIO 

1 

1 

.688 

* 

PLT  - 

54.26 

+  .74  ELT 

80.64 

141.43 

I 

2 

.750 

PLT  - 

74.18 

+  .53  ELT  +  .10 

SUP 

73.76 

100.32 

1 

3 

.761 

* 

PLT  - 

54.94 

+  .52  ELT  +  .08 

SUP  +  7.85 

SUPL0G72.61 

71.01 

2 

1 

.661 

* 

PLT 

a 

60.01  +  .75  ELT 

68.78 

79.84 

2 

2 

.683 

PLT 

m 

57.95  +  .64  ELT  +8.2  SUPLOG 

67.24  . 

44.65 

2 

3 

.697 

* 

PLT 

a 

47.86  +  .67  ELT  -  .02  SUP  +  11.7 

SUPL0G66 .34 

31.84 

2 

4 

.708 

PLT 

- 

88.27  +  .70  ELT  -  02  SUP  +  9.47  SUPLOG 

-  6.34  QTYLOG  65.68 

25.12 

2 

5 

.716 

* 

PLT 

m 

123.46  +  .68  ELT  -  .03  SUP  +7.9 
-  7.6  QTYLOG  -  3.3  EXT LOG 

SUPLOG 

65.25' 

20.83 

3 

1 

.  .427 

* 

PLT 

m 

132.81  +  .49  ELT 

102.39 

261.64 

3 

2 

.443 

PLT 

m 

45.12  +  .30  ELT  +  25.0  ELTLOG 

101.56 

/ 143.13 

3 

3 

.454 

PLT 

m 

27.04  +  .25  ELT  +  26.9  ELTLOG 
+5.86  SUPLOG 

100.95 

101.62 

3 

4~ 

.456 

* 

PLT 

a 

27.18  +  .26  ELT  +  26.8  ELTLOG 

f\r\*  /■>  *•+>  i  r  A  r> »  t-v  o  r» 

~  .Ovi  &U4T  T  J  *  Oufuvvl 

•  SJ  i 

/  /  •  Ov 

r3  * 

•  4 

1 

.693 

w 

PLT 

■* 

74.37  +  .61  ELT 

76.51 

290.55 

4 

2 

.719 

PLT 

a 

38.22  +  .57  ELT  +  9.57  QTYLOG 

73.78 

168.54 

4 

3 

.737 

PLT 

a 

2.78  +  .55  ELT  +  8.96  QTYLOG 
+7.05  EXTLOG 

71.88 

124.33 

*  4 

'i 

4 

.755 

*PLT 

!■ 

.05  +  .53  ELT  +7.6  QTYLOG  +  .00012 
EXT  +8.45  EXTLOG 

69.88 

103.46 

/. 

■  5 

.758 

PLT 

H 

4.3  +  .52  ELT  -  .01  QTY  +  10.1  QTYLOG 
+  .00012  EXT  +  8.2  EXTLOG 

69.63 

84.01 

5 

1 

.318 

* 

PLT 

m 

159.22  +  .58  ELI 

191.01 

31.42 

5 

2 

.436 

.  * 

PLT 

m 

99.69  +  .38  ELT  +  27.44  SUPLOG 

181.58 

32.79 

5 

3 

.450 

* 

PLT 

m 

-  63.64  +  .09  ELT  +  43.88  ELTLOG 
+28.03  SUPLOG 

180.53 

23.54 

> 

*  Equations  tested  further 
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TABLE  4' 


RESULTS  TO  BMP  -  STEPWISE  REGRESSION 
•  PM-CODE  GROUPING 


■M 

STEP 

MULT. 

R 

EQUATION 

STD. 

ERROR 

F 

RATIO 

.11 

1 

.510 

* 

PLT 

k» 

100.95  +  .624  ELT 

115.52 

71.5 

2 

.529 

if 

PLT 

■a 

82.37  +  .575  ELT  +9.25  SUPLOG 

114.02 

39.5 

3 

.537 

* 

PLT 

m 

24.35  +  .411  ELT  +  17.92  ELTLOG 
+9.86  SUPLOG 

113.31 

27.5 

4 

.539 

PLT 

m 

26.32  +  .40  ELT  +  18.06  ELTLOG  • 
+8.85  SUPLOG  +  .00003  EXT 

113.17 

20.8 

1&2 

1 

.684 

if 

PLT 

ta 

56.07  +  .75  ELT 

,  75.92 

23.0 

2 

.710 

★ 

PLT 

m 

40.35  +  .67  ELT  +  10.08  SUPLOG 

73.42 

13.3 

3&4 

1 

.525 

★ 

'  PLT 

m, 

109.07  +  .56  ELT 

98.77 

56.6 

2 

.540 

PLT 

m 

47.44  +  .36  ELT  +  20.47  ELTLOG 

97.65 

30.7' 

3 

.545 

* 

PLT 

m 

34.60  +  .34  ELT  +  21.71  ELTLOG 
+  4.24  SUPLOG 

97.32 

31.0 

4 

.547 

PLT 

m 

24.30  +  .34  ELT  +  20.64  ELTLOG 

+  5.07  SUPLOG  +  2.5S  QTYLOG 

97.22 

15.9 

1&3 

1 

.486 

ie 

PLT 

aa 

115.09  +  .55  ELT 

101.12 

41.1 

2 

.496 

ft  • 

ri/i 

M 

iuj.jo  t  .52  ELT  +  5.85  SUPLOG 

< 

3 

.506 

★ 

PLT 

KS 

36.14  +  .37  ELT  +  17.40  ELTLOG 
+  6.12  SUPLOG 

99.87 

15.2 

264 

i 

.687 

*PLT 

« 

73.49  +  .63  ELT 

74.77 

37.5 

2 

.701 

*PLT 

0 

72.98  +  .61  ELT  +  .00011  EXT 

73.43 

20.3  . 

3  ‘ 

.710 

PLT 

H 

49.04  +  .60  ELT  +  .00012  EXT 
+  4.20  EXTLOG 

72.66 

14.1  . 

4 

.715 

PLT 

m 

33.15  +  .59  ELT  +4.15  QTYLOG 

+  .00011  EXT  +  3.91  EXTLOG 

72.15 

10.9 

5 

.719 

*PLT 

m 

30.07  +  .59  ELT  -  .003  QTY  +  5.26 
QTYLOG  +  .00011  EXT  +  4.08  EXTLOG 

71.82 

8.9 

Groupings  -  (1&2)  ■  Competitives,  (3&4)  ■  Dir  Purch  Mftr. 
(1&3)  •  Alreadies,  (2&4)  «  1st  Tinea 


*  Equations  Tested  Further 
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REDUCTION  IN  ERROR  BY  TYPE  MODEL 


